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Abstract. 

Wikis  are  increasingly  used  in  diverse  settings  to  organize  large  amounts  of  information  into 
concise  articles  and  to  enable  broad  feedback  and  proposed  changes  to  that  information.  The 
BKCASE  Project  recently  released  a  draft  of  the  Guide  to  the  Systems  Engineering  Body  of 
Knowledge  (SEBoK)  as  a  wiki  (www.sebokwiki.org).  To  the  authors’  knowledge,  this  is  the 
first  time  a  wiki  has  been  used  as  the  underlying  technology  for  a  global  community  to 
develop  what  is  hoped  to  be  the  definitive  taxonomy,  vocabulary,  and  guide  to  knowledge  in 
a  discipline.  This  paper  describes  the  approach  taken  by  the  authors  in  setting  up  the  wiki,  the 
wiki’s  governance  process,  how  authors  used  the  wiki  collaboratively  to  create  the  latest  draft 
of  the  SEBoK,  how  the  wiki  is  being  used  by  SEBoK  reviewers  from  around  the  world,  a  set 
of  lessons  learned  from  adoption  to  implementation,  and  the  positive  impact  the  wiki  is 
having  on  both  the  quality  of  the  SEBoK  and  the  speed  with  which  the  SEBoK  is  maturing. 
Analogous  efforts  to  create  standards,  reference  curricula,  and  other  community  documents 
should  consider  adopting  a  wiki  as  their  underlying  technology  together  with  the  appropriate 
governance  model  to  enable  effective  authoring  and  review. 


Background 

In  2009,  the  Stevens  Institute  of  Technology  and  the  Naval  Postgraduate  School  began  the 
Body  of  Knowledge  and  Curriculum  to  Advance  Systems  Engineering  (BKCASE™)  project. 
The  BKCASE  project  has  four  main  objectives  (BKCASE  2009);  the  first  of  these  objectives 
is  to  create  a  systems  engineering  body  of  knowledge  (SEBoK)  which  will  be  globally 
recognized  by  the  SE  community  as  the  authoritative  guide  to  the  body  of  knowledge  for  the 
systems  engineering  (SE)  discipline.  The  utilization  of  a  wiki  platform  for  development  and 
delivery  of  the  SEBoK  is  intended  to  support  this  objective,  making  the  SEBoK  globally 
available  and  providing  opportunities  for  global  review  of  draft  versions. 


SEBoK  Development 

The  BKCASE  author  team  developed  the  SEBoK  in  four  iterations  labeled  SEBoK  versions 
0.25,  0.5,  0.75,  and  1.0  (see  Figure  1,  below).  Version  0.25,  a  prototype,  was  released  for 
limited  review  to  300  members  of  the  SE  community  from  September  -  December  2010. 
Version  0.5  is  the  current  version  of  the  SEBoK,  and  is  open  for  global  review  until 
December  2011.  [Note:  This  will  be  updated  after  the  close  of  the  review  process  in 
December  2011.]  The  next  versions  of  the  SEBoK  will  be  released  in  spring  2012  and  fall 
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2012,  respectively.  Version  1.0  will  be  the  final  version  under  the  BKCASE  project;  it  is 
anticipated  that  INCOSE  and  the  IEEE  Systems  Council  will  take  joint  stewardship  of  the 
SEBoK  after  the  release  of  version  1.0.  [Note:  This  will  be  updated  to  reflect  progress  on 
SEBoK  0.75  when  the  paper  is  finalized]. 


The  BKCASE  project  is  staffed  by  almost  70  authors  from  5  continents.  These  authors  meet 
at  quarterly  meetings  to  discuss  the  project,  the  way  ahead,  and  to  conduct  working  sessions. 
(See  Figure  1,  below). 
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Figure  1 .  Development  Schedule  for  SEBoK 

Why  Wiki? 

The  prototype  version  of  the  SEBoK,  0.25,  was  release  for  limited  review  as  a  traditional 
document.  This  aligns  with  other  bodies  of  knowledge  examined,  such  as  the  Software 
Engineering  Body  of  Knowledge  (SWEBOK)  (Abran  et  al.  2004)  and  the  Project 
Management  Body  of  Knowledge  (PMBOK™)  (PMI  2008).  However,  these  prototype 
versions,  which  was  believed  to  contain  only  a  small  percentage  of  the  final  SEBoK  materials, 
was  over  600  pages.  In  addition,  there  was  little  functionality  to  version  0.25;  both  the 
reviewers  and  the  professional  societies  -  INCOSE  and  IEEE  CS  -  expressed  interest  in 
improving  the  functionality  of  the  SEBoK.  This  led  to  a  list  of  desired  improvements  for  the 
SEBoK,  including  better  navigation,  improved  searching,  recommendations  for  related  topics, 
improved  linkages  between  content,  improved  definition  and  consistency  of  terminology,  and 
easier  access  to  references  included  in  the  SEBoK. 

The  authors  considered  several  different  ways  in  which  these  improvements  might  be 
implemented  with  consideration  for  the  BKCASE  constraints  mandated  by  the  project 
sponsor.  First,  all  materials  developed  for  BKCASE  must  be  made  available  at  no  charge 
worldwide.  Second,  provide  the  broader  SE  community  with  an  improved  mechanism  for 
providing  feedback  to  ensure  that  the  SEBoK  will  meet  the  objective  of  being  globally 
accepted.  In  addition,  as  a  body  of  knowledge,  the  SEBoK  must  have  some  level  of  stability; 
this  is  seen  in  the  SWBOK  and  PMBOK,  which  are  updated  after  a  period  of  several  years. 
Finally,  it  was  determined  that  in  planning  for  future  versions  of  the  SEBoK  (post  1.0),  it 
would  be  useful  to  provide  a  mechanism  for  collecting  community  feedback  that  would 
support  revisions. 

Several  options  were  considered  for  enabling  these  capabilities,  including  the  use  of 
traditional  websites/web  forums  hyperlinked  PDF  documents,  and  several  different  types  of 
wiki.  At  the  January  2011  workshop,  the  author  team  agreed  that  a  wiki  implementation  was 
the  most  appropriate.  The  authors  determined,  however,  that  a  traditional  wiki  which  could 
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be  edited  freely  by  the  public  was  not  an  appropriate  model  for  the  SEBoK.  Therefore,  to 
maintain  credibility  it  was  important  to  develop  within  a  wiki  environment  that  could  be 
restricted  (or  opened),  as  appropriate,  and  provide  a  variety  of  access  control  options.  Balance 
between  an  open  and  collaborative  environment  for  publication  and  a  restricted  environment 
during  development  is  an  important  aspect  of  using  wiki  technology.  After  examining  many 
different  options,  the  team  agreed  to  utilize  the  MediaWiki  engine,  along  with  the 
implementation  of  many  different  access  controls  and  technical  adjustments  that  were  made 
as  needed  to  create  a  viable  and  appeasing  environment  for  the  users  throughout  the  project 
life  cycle. 


Wiki  Development  of  a  Body  of  Knowledge 

SEBoK  0.5  was  developed  primarily  between  April  and  September  2011.  The  review 
comments  from  the  0.25  version  were  incorporated  into  the  revisions. 

Again,  it  was  critical  that  the  wiki  be  a  ‘closed’  environment  during  development  -  allowing 
the  author  team  a  space  to  iterate  draft  materials  without  outside  alterations.  However,  access 
had  to  be  controlled  without  losing  many  of  the  benefits  of  a  wiki  environment,  such  as  the 
ability  to  see  updates  in  real  time  worldwide  and  to  provide  feedback  on  others’  work.  Also, 
as  the  SEBoK  does  represent  a  body  of  knowledge,  it  was  critical  that  there  be  some  stability, 
at  least  in  the  architecture,  to  enable  author  to  improve  integration  between  related  elements 
of  the  SEBoK.  To  that  end,  a  mixture  of  technology-based  and  process-based  solutions  was 
used  during  the  development. 


Development  Control 

For  the  development  of  SEBoK  0.5,  the  ability  to  view,  edit,  and  create  accounts  was 
restricted.  This  created  a  wiki  environment  which  was  private  -  accessible  only  to  the  author 
team.  However,  once  an  author  (or  other  authorized  user)  was  inside  the  space,  all  of  the 
other  functionalities  were  available  -  authors  in  the  wiki  were  able  to  edit  any  and  all  pages. 
However,  the  BKCASE  author  team’s  staffing  allocation  did  not  include  all  authors  working 
on  all  articles  at  all  times.  In  order  to  reconcile  author  assignments  and  open  access,  the 
BKCASE  team  agreed  to  a  process  for  controlling  development.  Authors  were  on  their  honor 
to  directly  edit  only  the  areas  of  the  wiki  assigned  to  them.  They  were  instructed  to  provide 
feedback  on  any  other  aspects  of  the  wiki  using  the  MediaWiki  discussion  feature.  This 
allowed  authors  some  control/autonomy  during  initial  drafting,  while  still  permitting  and 
encouraging  all  authors  to  provide  feedback  on  any  article. 

Configuration  Management 

To  ensure  some  stability  of  the  architecture  of  the  wiki,  structural  changes  to  the  wiki  -  the 
creation,  deletion,  or  renaming  of  pages  -  were  restricted  to  a  limited  number  of  leadership 
members.  However,  because  it  was  important  for  authors  to  take  ownership  of  the  different 
parts  of  the  wiki,  a  fairly  traditional  configuration  management  process  was  implemented  to 
allow  them  to  propose  architectural  changes.  The  process  included  initial  approval  by  a  peer 
group  -  working  on  the  same  group  of  pages  -  before  a  change  was  sent  to  the  configuration 
management  board. 


Structure 

SEBoK  0.5  is  the  rough  equivalent  to  1,000  pages  in  a  traditional  document.  Though 
breaking  the  SEBoK  down  into  individual  wiki  pages  improves  manageability,  a  collection  of 
web-based  pages  is  not  sufficient  for  a  body  of  knowledge.  To  remedy  this  the  BKCASE 


3 


authors  identified  a  way  to  organize  the  information  contained  in  the  SEBoK  into  three  main 
units:  parts,  knowledge  areas,  and  topics.  A  “part”  is  the  highest  order  grouping  in  the 
SEBoK.  There  are  seven  parts  in  version  0.5;  the  parts  cover  introductory  materials  for  the 
SEBoK,  the  nature  of  systems,  an  overview  of  SE,  the  applications  of  SE  to  different  types  of 
systems,  the  utilization  of  SE  within  an  organization,  the  interaction  of  SE  with  other 
disciplines,  and  a  collection  of  real  world  SE  examples. 

Each  part  consists  of  a  collection  of  related  knowledge  areas  (KAs).  “Knowledge  Area”  is  a 
term  commonly  found  in  bodies  of  knowledge  (e.g.  the  SWEBOK  and  PMBOK™)  that  is 
used  as  an  organizational  unit  for  related  information.  There  are  32  KAs  in  version  0.5; 
examples  include  “Types  of  Systems”,  “System  Definition”,  and  “Service  Systems 
Engineering”.  A  knowledge  area,  in  turn,  consists  of  a  set  of  topics.  A  topic  is  the  lowest 
organizational  level  of  the  SEBoK  and  is  a  concept  that  defines  specifically  related 
knowledge.  There  are  115  topics  in  version  0.5;  examples  include  “Classifications  of 
Systems”,  “System  Integration”,  and  “Software  Engineering  and  Systems  Engineering: 
Similarities  and  Differences.” 


Wiki  in  Review 

[Note:  This  will  be  updated  upon  the  close  of  the  review  process].  SEBoK  0.5  was  released 
for  unlimited  worldwide  review  on  September  19,  201 1  and  will  be  available  for  review  until 
December  15,  2011.  During  the  development  phase  of  the  wiki,  it  was  critical  that  only  a 
small  number  of  individuals  access  the  wiki  at  all,  but  that  those  individuals  have  almost 
complete  control.  During  the  review  period,  the  opposite  is  true:  anyone  on  the  planet 
should  be  able  to  access  the  wiki,  anyone  who  wants  should  be  able  to  provide  comments,  but 
the  actual  content  itself  -  the  pages  -  must  be  left  untouched  to  enable  a  consistent  review. 

For  the  review  period,  the  wiki  was  changed  from  “private”  to  “public”.  However,  in  order 
to  provide  a  stable  version  for  review,  it  was  critical  that  changes  not  be  made.  This  required 
protection  of  all  pages  of  the  wiki.  While  this  is  a  standard  feature  in  most  wikis,  the 
BKCASE  project  had  to  use  a  higher  level  of  protection  to  ensure  that  even  the  authors  would 
not  be  able  to  make  changes  during  the  review  period. 

As  mentioned,  a  primary  reason  for  utilizing  wiki  technology  was  to  provide  an  easy 
mechanism  for  the  community  to  provide  feedback  -  and  to  enable  the  individuals  to  see  the 
feedback  provided  by  others.  To  this  end,  the  LiquidThreads  -  threaded  discussion  - 
extension  has  been  used  for  SEBoK  0.5.  This  enables  anyone  who  desires  to  provide  reviews 
the  ability  to  input  their  feedback  and  review  and  respond  to  the  feedback  of  others. 

The  BKCASE  team  had  several  major  areas  of  interest  for  the  review  -  to  that  end,  they 
developed  a  series  of  prompts  that  are  incorporated  into  the  discussion  for  each  page.  These 
may  be  seen  in  Table  1.  The  rationale  for  providing  some  structure  for  feedback  was  to 
ensure  that  reviewers  are  aware  of  the  main  areas  of  concern,  while  also  enabling  flexibility 
in  their  responses  (e.g.  the  “Open  Discussion”  thread). 


Table  1 .  Discussion  Prompts  for  Content  Review  of  SEBoK  0.5 


Thread  Title 

Prompt 

Integration 

A  goal  of  the  SEBoK  is  to  ensure  that  concepts  are  discussed 
consistently  throughout.  Please  use  this  discussion  thread  to  provide 
feedback  on  integration  of  the  concepts  in  this  article  -  both  what 
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Thread  Title 

Prompt 

worked  well  and  what  can  use  improvement. 

Multiple 

Perspectives 

A  goal  of  the  SEBoK  is  to  provide  guidance  to  the  many  varying 
perspectives  on  each  topic.  Please  use  this  discussion  thread  to 
provide  feedback  on  the  perspectives  provided  and  additional 
perspectives  that  can  be  considered  for  this  topic.  Please  suggest 
references  in  the  "References"  discussion  to  support  these  additional 
perspectives. 

Terminology 

A  goal  of  the  SEBoK  is  to  ensure  that  terms  are  discussed 
consistently.  Please  provide  any  feedback  specific  to  the  terminology 
used  in  this  article  -  both  what  worked  well  and  what  can  use 
improvement. 

References 

The  SEBoK  is  designed  to  provide  pointers  to  a  variety  of  relevant 
works  within  the  systems  engineering  community  for  a  given  topic. 
For  version  0.5,  the  author  team  acknowledges  that  more  diversity  of 
references  is  still  needed.  Please  provide  your  feedback  on  references 
here,  specifically  what  additional  references  should  be  considered  for 
future  incorporation. 

General  Content 

Please  provide  your  general  impressions  of  the  article’s  content  and 
usefulness,  along  with  your  recommendations  for  improvement  here. 

Open  Discussion 

Please  place  all  other  comments  here. 

In  addition  to  ensuring  that  the  SEBoK  review  solicits  the  desired  feedback,  the  review 
approach  also  provides  an  opportunity  for  the  BKCASE  authors  to  get  feedback  on  their 
materials  in  real  time  to  begin  working  on  the  draft  of  SEBoK  0.75.  When  the  review  period 
for  0.5  is  complete,  the  authors  will  provide  a  summary  of  each  discussion  thread  in  the  wiki 
along  with  an  adjudication  -  the  actions  taken  by  the  authors  in  response  to  the  feedback. 
This  will  enable  a  transparent  adjudication  process  while  allowing  reviewers  to  understand 
how  their  efforts  are  shaping  the  SEBoK. 

[Note:  This  will  be  updated  to  reflect  key  lessons  learned  from  the  review  period  -  including 
level  of  community  response,  level  of  interaction  in  the  feedback  structures,  etc.  At  the  time 
of  final  paper  submission,  version  0.75  will  have  been  released.] 


Publication  and  Beyond 

It  is  anticipated  that  the  wiki  implementation  will  be  used  for  the  development  and  review  of 
SEBoK  0.75  and  for  the  development  and  final  delivery  of  SEBoK  1.0  in  fall  2012.  As 
previously  mentioned,  it  is  anticipated  that  INCOSE  and  the  IEEE  Systems  Council  will  take 
stewardship  of  the  SEBoK  at  this  time.  It  is  believed  that  the  professional  societies  will 
maintain  the  wiki  delivery  of  the  SEBoK,  as  it  will  provide  a  mechanism  for  collecting  and 
archiving  community  feedback  in  real  time  that  will  be  available  for  future  revisions. 

As  a  body  of  knowledge,  the  SEBoK  will  likely  maintain  permission  controls  similar  to  those 
currently  implemented  -  namely  that  for  each  published  version,  all  will  be  able  to  comment, 
while  few  will  be  able  to  edit.  Again,  the  rationale  for  this  is  that  a  body  of  knowledge 
requires  some  level  of  stability  over  time.  However,  it  is  hoped  that  due  to  the  manner  in 
which  community  feedback  is  collected,  the  professional  societies  will  be  empowered  to 
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make  updates  to  the  SEBoK  more  frequently  than  other  bodies  of  knowledge.  (For  example, 
the  SWEBOK  was  published  in  2004  and  is  only  now  undergoing  its  first  revision.) 

A  final  advantage  to  maintaining  wiki  implementation  is  that  while  the  content  of  the  SEBoK 
needs  to  be  somewhat  stable,  the  comments  do  not  need  to  be.  In  fact,  in  utilizing  the 
comment  functions,  it  is  possible  for  members  of  the  community  to  recommend  additional 
view  points,  new  literature,  or  introduce  an  emerging  area  of  research.  Again,  this  will  make 
the  SEBoK  more  valuable  to  the  community  as  a  whole,  even  while  maintaining  the 
consistency  required  of  a  body  of  knowledge. 

Conclusions 

Wiki  technology  is  inexpensive  -  “free”  in  the  case  of  MediaWiki-  easy  to  set  up,  easy  to 
use,  and  enables  large,  geographically  dispersed  groups  to  work  on  complex  projects  in  real 
time,  with  the  capability  for  instantaneous  two-way  communication  viewable  by  a  limited  set 
of  users  or  the  public.  It  can  be  tightly  controlled  so  that  only  certain  people  can  even  see  the 
information  or  it  can  be  completely  open,  with  anyone  having  the  ability  to  edit.  The 
BKCASE  author  team  has  successfully  utilized  wiki  technology  as  a  collaborative  tool;  it  is 
believed  that  this  tool  will  shape  the  way  that  the  SE  community  shares  information  and 
interacts  going  forward. 


Possible  Future  Research 

The  SEBoK  is  an  example  of  the  utilization  of  a  wiki  for  developing  publications.  How  else 
are  they  being  used?  For  example,  in  disaster  response,  communications  between  first 
responder  organizations  and  development  of  a  common  operating  picture  are  consistently 
referenced  as  recurring  a  problem.  Could  wiki  tech  provide  a  fast  and  easy  way  for  updates 
when  physical  infrastructure  survives?  Could  you  wiki  technology  be  used  for  enabling 
communication  in  and  simultaneously  documenting  a  SE  effort?  The  possibilities  are 
intriguing  -  future  research  efforts  may  focus  on  how  this  type  of  Web  2.0  technology  may 
be  best  employed  in  practice. 
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